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Abstract 

We describe a quantum PAC learning algorithm for DNF formulae under the uniform dis- 
tribution with a query complexity of 0(s 3 /e + s 2 /e 2 ), where s is the size of DNF formula and e 
is the PAC error accuracy. If s and 1/e are comparable, this gives a modest improvement over 
a previously known classical query complexity of 0(ns 2 /e 2 ). We also show a lower bound of 
tt(s logn/n) on the query complexity of any quantum PAC algorithm for learning a DNF of size 
s with n inputs under the uniform distribution. 

1 Introduction 

In this abstract we describe a quantum learning algorithm for DNF formulae under the uniform 
distribution using quantum membership queries. Although Bshouty and Jackson [Q] have shown 
that it is possible to adapt Jackson's Harmonic Sieve algorithm Q to the quantum setting, our 
goal is different. We will focus on reducing the number of quantum membership queries used by 
the DNF learning algorithm whereas their motivation was in showing that quantum examples are 
sufficient for learning DNF. 

The Harmonic Sieve HS algorithm combines two crucial independent algorithms. The first 
algorithm is an inner algorithm for finding parity functions that weakly approximate the target 
DNF function. The second algorithm used in the Harmonic Sieve is an outer algorithm that is a 
boosting algorithm. A weak learning algorithm is an algorithm that produces hypotheses whose 
accuracy are slightly better than random guessing. Boosting is a method for improving the accuracy 
of hypotheses given by a weak learning algorithm. 

For the inner algorithm, a Fourier-based algorithm given in JiT| (called the KM algorithm) is 
used in HS for finding the weak parity approximators. The KM algorithm is based on a similar 
method given by Goldreich and Levin jS] in their seminal work on hardcore bits in cryptography. 
Subsequently, Levin [12| and Goldreich ||], independently, gave highly improved methods for solving 
this so-called Goldreich- Levin problem. Their ideas were adapted by Bshouty et al. || to obtain 
a weak DNF learning algorithm with query and time complexity of 0(n/7 2 ), where 7 is the weak 
advantage of the parity approximator. By a result of Jackson §§, 7 = 0(1/ s) for DNF formula of 
size s. 

For the outer algorithm, the original HS used a boosting method of Freund ||] called Fl that 
has various nice features. Recently, Klivans and Servedio [10] observed that a construction of 
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Impagliazzo || gave a smoother boosting algorithm called IHA. It was shown that IHA is a 0(1/ e)- 
smooth 0(7~ 2 e~ 2 )-stage boosting algorithm, where 7 is the weak advantage of the weak learning 
algorithm and e is the target accuracy. In contrast, Fl is a 0(l/e 3 )-smooth 0(7~ 2 log(l/e))-stage 
boosting algorithm. 

The fastest known algorithm for learning DNF is obtained by combining the two improved 
independent components that results in a total running time of 0(ns 4 /e 2 ) and a query complexity 
ofO(ns 2 /e 2 ) 1,0. 

We describe an efficient quantum DNF learning algorithm by combining a quantum Goldreich- 
Levin algorithm QGL of Adcock and Cleve [|J with a well-known highly efficient boosting algorithm 
of Freund called Bc om b 0]. The quantum algorithm of Adcock and Cleve used only 0(1/7) queries 
(beating a classical lower bound of f2(n/7 2 ) proved also in [U). Freund's Bc om b algorithm is 
a 0(l/e)-smooth 0(7~ 2 log(l/e))-stage boosting algorithm. After adapting both algorithms for 
quantum PAC learning, we obtain a quantum Harmonic Sieve algorithm QHS with a sample com- 
plexity of 0(s 3 /e + s 2 /e 2 ). In contrast to the best known classical upper bound of 0(ns 2 /e 2 ), this 
gives a modest improvement if s and 1/e are comparable. 

As shown in Q, the quantum Goldreich-Levin algorithm has applications to quantum cryptog- 
raphy. In this work, we show one of its applications in computational learning theory. 

For the sake of exposition, in this abstract we will describe our quantum DNF PAC learning 
algorithm using a conceptually simpler boosting algorithm SmoothBoost given by Servedio |l3|]. We 
describe a boost-by-filtering version of Servedio's SmoothBoost that is a 0(l/e)-smooth 0(7 _2 e -1 )- 
stage boosting algorithm. So, we incur an extra 1/e factor in the sample complexity. We defer the 
details of using Bc mb in QHS to the final version of this paper. 

Finally, we prove a query lower bound of f2 (s log n/n) on any quantum PAC learning algorithm 
for DNF under the uniform distribution with (quantum) membership queries. 

2 Preliminaries 

We are interested in algorithms for learning approximations to an unknown function that is a 
member of a particular class of functions. The specific function class of interest in this paper is 
that of DNF expressions, that is, Boolean functions that can be expressed as a disjunction of terms, 
where each term is a conjunction of Boolean variables (possibly negated). Given a target DNF 
expression / : {0, l} n — > { — 1,+1} having s terms along with an accuracy parameter < e < 1/2 
and a confidence parameter 5 > 0, the goal is to with probability at least 1 — 5 produce a hypothesis 
h such Pr Xr ^u n [f(x) 7^ h(x)\ < e, where U n represents the uniform distribution over {0, l} n . We 
will sometimes refer to such an h as an e- approximator to /, or equivalently say that h has \ — e 
advantage (this represents the advantage over the agreement between / and a random function, 
which is 1/2). A learning algorithm that can guarantee only 7 > advantage in the hypothesis 
produced but can do so with arbitrarily small probability of failure 5 is called a weak learning 
algorithm, and the hypothesis produced is a weak approximator. 

The information our learning algorithm is given about the target function varies. One form is 
a sample, that is, a set S of input/output pairs for the function. We often use x to denote an input 
and f(x) the associated output, and x £ S to denote that x is one of the inputs of the pairs in 
S. Another type of information we sometimes use is a membership oracle for /, MEMf. Such an 
oracle is given an input x and returns the function's output f(x). 
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Input: Parameters 0<e<l/2, 0<7<l/2 
Sample S of target / 
Weak learning algorithm WL 

Output: Hypothesis h 

1. Us = the uniform distribution over S 

2. Mi(x) = l,Vs€S 

3. 2V (x) = 0, Vx g 5 

4. 0^ 7/(2 + 7) 

5. t «- 1 

6. while E x „u 3 [M t (x)] > e do 

7. = M^^mE^^x)]), Vx e 5 

8. ^^WL(S,A^ = ^(e~V 2 )) 

9. iV t (x) = iV t _i(s) + f{x)h t {x) - 9, Vx e 5 

10. Mt+i(s) = [iV t (x) < 0] + (1 - 7) JV *(*)/ 2 [iV t (x) > 01, VxGS 
11. 

12. end while 

13. T«-t-l 

14. H^^li^ 

15. return /i = sign(//). 



Figure 1: The SmoothBoost algorithm of Servedio |l 

3 A smoother Boost-by-Filtering algorithm 



A modification of Servedio's SmoothBoost boosting algorithm [ 13 1 is described in this section. A 
special case (discrete weak hypotheses and fixed margin) version of SmoothBoost sufficient for our 
purposes is shown in Figure |]. SmoothBoost is a boosting-by-sampling method that can be applied 
to a weak learning algorithm in order to produce a hypothesis that closely approximates the sample. 
Specifically, SmoothBoost receives as input a sample S of size m as well as accuracy parameter e. It 
is also given a weak learning algorithm WL. The boosting algorithm defines a series of distributions 
Dt over S and successively calls the weak learning algorithm, providing it with the sample S and 
with one of the distributions Dt- In the end, the algorithm combines the weak hypotheses returned 
by the calls to the weak learner into a single hypothesis h. 
Servedio proves three key properties of SmoothBoost: 

Lemma 1 (Servedio) Let f be a target function, and let S, e, 7, h, and Dt be as defined in 
Figure [I]. Then 

1. If every weak hypothesis ht returned by WL has advantage at least 7 with respect to Dt, then 
SmoothBoost will terminate after T = 0(e _1 7~ 2 ) stages. 

2. //SmoothBoost terminates, then Vx x ^u s [f{x) 7^ h{x)\ < e, where Us represents the uniform 
distribution over S (Servedio actually proves a stronger margin result that implies this). 

3. Loo^mDt) < 1/e for all t, where m = \S\ (this is the smoothness property of SmoothBoost ). 
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Here we adapt this algorithm to obtain a boosting-by-filtering algorithm that will be used by 
the Harmonic Sieve. First, notice that Lemma [l] holds for the special case S = {0, l} n . However, 
there are potential problems with running the SmoothBoost algorithm directly on such a large S. 
First, it is not computationally feasible to exactly compute E xr ^u n [M t (x)], where U n represents the 
uniform distribution over {0, l} n . So instead we must estimate this quantity by sampling. This 
has a small impact on both the form of the loop condition for the algorithm (line |6|), but also on 
the "distributions" Dt passed to the weak learner (line 0). In fact, the Dt that will be passed to 
the weak learner will generally not be a true distribution at all, but instead a constant multiplied 
by a distribution due to the constant error in our estimate of Ei x ^u n [M t (x)]. 

We will deal with the weak learner later, so for now let us assume that the weak learner produces 
the same hypothesis ht given an approximation to Dt as it would given the actual distribution. Then 
notice that the computations for Nt and Mt+\ are unchanged, so the only impact on the boosting 
algorithm has to do with the loop condition at line []. This is easily addressed: let Et represent an 
estimate of ~E x ~U n [M t (x)] to within additive error e/3 and change the loop condition to E t > 2e/3. 
Then if the loop terminates it must be that ~Ei x ^jj„ [Mt(x)] < e, as before. It is easily verified 
that given this condition, Servedio's proof implies that h is an e-approximator to / with respect 
to the uniform distribution. Furthermore, since ~Ei x ^u n [Mt(x)] > e/3 if the algorithm terminates, 
the other statements of Lemma p] change only by constant factors. In particular, the smoothness 
condition of the lemma now becomes L 00 {2 n Dt) < 3/e for all t. 

Finally, because O < Mt(x) < 1 for all t and x, the Hoeffding bound gives that taking the 
sample mean of Mt(x) over a sample of size f2(e -2 ) will, with constant probability, produce an 
estimate with additive error at most e/3. Furthermore, if the algorithm terminates in T steps, then 
a single uniform random sample R of size il(log(T)/e 2 ) guarantees, with constant probability, that 
estimating the expected value of M t (x) by the sample mean over R at every step t will produce an 
e/3 accurate estimate at every step. 

Figure [2] presents the modified SmoothBoost algorithm. Notice that in place of a sample S 
representing the target function /, we are assuming that we are given a membership oracle MEMj. 
We will subsequently consider quantum versions of this algorithm and of the membership oracle. 
For this reason, we show the definitions of M and N as being over all of {0, l} n , although for a 
classical algorithm the only values that would actually be used are those corresponding to x € R. 

While the SmoothBoost algorithm has been presented for illustration, Klivans and Servedio |]io| 
have shown that one of Freund's boosting algorithms, which they call Bc om b> is actually slightly 
superior to SmoothBoost for our purposes. Specifically, they note that Bq 0II1D has properties similar 
to those of SmoothBoost given in Lemma EL with the change that the number of stages T improves 
from 0(e _1 7~ 2 ) to 0(log(l/e)/7 2 ) while the smoothness of each of the distributions D t passed to 
the weak learner satisfies (when learning over all of {0,1}") L 00 (2 n Dt) = 0(log(l/e)/e). we will 
continue to use SmoothBoost in our analysis here, since BQ omD and its analysis are noticeably more 
complicated than SmoothBoost and its analysis. However, our final sample size bounds will be 
stated as if Bc omD is being used, and the final version of this paper will include details of the Bc mb 
analysis. 

4 A query-efficient quantum WDNF algorithm 

In this section we describe a quantum weak learning algorithm WDNF for finding parity approxima- 
tors of non-Boolean functions under smooth distributions. This algorithm is based on a quantum 
Goldreich-Levin algorithm given by Adcock and Cleve jjj]. For completeness we describe the quan- 
tum Goldreich-Levin algorithm in the following. This algorithm utilizes the Pauli X (complement) 
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Input: Parameters 0<e<l/2, 0<7<l/2 
Membership oracle MEMf 
Weak learning algorithm WL 

Output: Hypothesis h 

1. Draw uniform random sample R of f2(log(e _1 7 _1 )/e 2 ) instances x and label using MEMf 

2. Ur = the uniform distribution over R 

3. Mi(x) = 1, Vx G {0, 1}™ 

4. JVb(x) = 0, Vx G {0, 1}™ 

5. 0^7/(2 + 7) 

6. i <- 1 

7. while E^^fM^x)] > 2e/3 do 

8. D t (x) = M t {x)/(2 n B x ^ UR [M t (x)]) 

9. ht ^VL(MEM f ,D t ,5 = n(e- 1 -f- 2 )) 

10. iV<(x) = Nt-i(x) + f{x)h t {x) - 0, Vx G {0, 1}" 

11. M t+1 {x) = \N t {x) < 0] + (1 - 7)^W/ 2 \N t {x) > 01, Vx G {0, 1}" 

12. tt-t+1 

13. end while 

14. r <— t — i 

15. H=±Yl=ihi 

16. return /i = sign(.H'). 

Figure 2: The SmoothBoost modified for boost-by-filtering, 
and Z (controlled phase flip) gates and the Hadamard gate H defined as follows. 

x= (? o)' z= G ff= ^G -i 

Let iif n = ii"® n (n-fold tensor of H with itself) be the Walsh-Hadamard transform on n qubits and 
let Uo{Ylx a x\ x )) = S-r^o a x\ x ) ~ «o|0) be the unitary transformation that flips the phase of the 
all-zero state. 

The Umq transformation that represents a noisy membership oracle with respect to a parity 
function xa defined in H is given by 



U M Q\x)\0 m ) = a x \x,u x ,A- x) + p x \x,v x ,A- x), 

where a x > 1/2+7 and Y^ x P x — 1/2—7- By a result of Jackson for any DNF formula / with 
s terms, there is a parity function A such that Pr[/(x) = xa(^)] > 1/2 + 7/2, for 7 = l/(2s + 1). 
Thus, a noiseless DNF oracle QMQf is a noisy oracle Umq for some parity function \A- Thus, we 
may assume that Umq is a unitary transformation that represents a quantum membership oracle 
QMQf for a DNF formula / that maps |x)|0 m ) to \x)\u x , f(x)), for some string u x G {0, l} m_1 
that represents the work space of the oracle. 

The quantum algorithm QGL of Adcock and Cleve is represented by the following unitary trans- 
formation 

C={H n ® I m+ i)(U M Q ® /i)(/ n+m _i &> Z){U M q ® h){H n ®I m ®X) (1) 
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Input: Parameters n, 7 € (0, 1/2), 5 > 

Quantum membership oracle QMQf for Boolean function / 

represented by a unitary tranformation ?7mq 
Random uniform sample R of size f2(7 -2 log(l/<5)). 
Output: A coefficient A with the property that Pvo[f = Xa] > 5 + 7 
with probability at least 1 — 5. 

1. Let C be defined as in Equation [j]. 

2. Label using QMQf. 

3. Define a sampling-based Ueq as in Equation [2]. 

4. \<p) <- C|0 n ) 7 |0 m ) A |0) B 

5. for fc = 1,... ,0(1/7) do 

6. |p> <- -CiJoC^BQb) 

7. end for 

8. Measure and return the contents of register I. 



Figure 3: The quantum weak learning algorithm QWDNF for uniform distribution. 

applied to the initial superposition of |0 n ,0 m ,0). In |jj it was proved that the quantum algorithm 
QGL prepares a superposition of all n-bit strings such that the probability of observing the coefficient 
A is 47 2 . By repeating this for 0(l/7 2 ) stages, we can recover A with constant probability. 

The number of stages can be reduced to 0(1/7) by using a technique called amplitude amplifi- 
cation. This amplification technique uses an iterate of the form 

G = (-CU C^U EQ ) k C\0 n ,0 m ,0), 

where k is approximately 0(1/7), and Ueq is a unitary transformation that represents a quantum 
equivalence oracle QEQ f . The transformation U eq is defined as 

UeqW) = j |a J a) ^J2^' " 9 (2) 

For the purpose of learning DNF, we need to simulate Ueq using a sampling algorithm that has 
access to QMQf. A classical application of Hoeffding sampling requires f2(l/7 2 ) queries to QMOjQ. 
To simulate Ueq, we will simply use a sample R of size 0(l/7 2 log(l/<5)) to obtain a good estimate 
with probability at least 1 — 5. 

Finally, recall that we will be applying boosting to this weak learning algorithm, which means 
that QWDNF will be called a number of times. However, it is not necessary to draw a new random 
sample R each time QWDNF is called, as the boosting algorithm merely wants a guarantee that 
the algorithm succeeds with high probability and does not require independence. The resulting 
quantum weak learning algorithm for DNF, which we denote QWDNF, is described in Figure ||. 

4.1 Non-Boolean Functions over Smooth Distributions 

Recall that in the Harmonic Sieve algorithm |]], we need to find weak Parity approximators for 
non-Boolean functions g that is based on the DNF formula / and the current boosting distribution 

1 Grover has proposed a quantum algorithm for estimating the mean that requires log log —) queries. However, 
in our setting, we will use fewer queries if we estimate this value classically because we can use a single sample for 
all estimates, as discussed below. 
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D in SmoothBoost, i.e., we need to consider expressions of the form (we have dropped subscripts 
for convenience) 

B D [f(x) XA (x)} = ^TD(x)f(x) XA (x) 

x 

= ^2«E[M(x)] /(x)XA(x) 

B[M(x)f(x) XA (x)} 
B[M(x)} 

This shows a reduction from finding a coefficient A such that | Ex? [/"x^l] I I s large to finding a 
coefficient A so that |E[/[gXyl]|> where g(x) = M(x)f(x), is large. Assuming that E[M(x)] > e/3, 
we will use the algorithm QWDNF to find a coefficient A such that for some constant C2 

\E[M(x)f(x) X A(x)}\> m J 6 + =T. 

Note that < M{x) < 1, for all x. Thus we can use a technique of Bshouty and Jackson [|| that 
transforms the problem to the individual bits of M{x). Let d = log(3/T), where T is as above. Let 
a(x) = [2 d M(x)\ /2 d , i.e., M(x) truncated to include only d of its most significant bits. Assume 
that a = Y$=i a 3 2 ~ j + k2 ~ d i where a j e {-!> *} and k G {-1>0, 1}- Th us 

\B[M(x)f(x) XA (x)]\ - ^ < \E[a(x)f(x)xA(x)}\ < max\E[a j (x)f(x) X A(x)]\ + ^ 

O JO 

thus there exists j so that |E[aj(x)/(x)xA(£)]| > T/3, assuming \~E[M (x) f (x) X a(x)]\ > T. 

Note that to simulate Ueq for verifying that the non-Boolean function g(x) = M(x)f(x) has a 
T-heavy coefficient at A, i.e., |<?(^4)| > T, we need a sample of size at least 1/T 2 ~ (s/e) 2 . 

5 A quantum Harmonic Sieve algorithm 

In this section, we describe a quantum version of the Harmonic Sieve algorithm obtained by com- 
bining the quantum Goldreich-Levin algorithm and the SmoothBoost boosting algorithm (see Fig- 
ure |). 

The top level part of this algorithm involves 0(s 2 /e) boosting rounds^] and each round requires 
invoking the algorithm QWDNF that uses 0(s/e) queries. The "oracle" QMQf ■ Dt represents the 
procedure that will produce Boolean functions representing the bits of Mtf and simulate quantum 
membership oracles to be passed to QWDNF. There is an additional cost of a random sample of size 
0(s 2 /e 2 ) for estimating the expression E[Mj] to within O(e) and for simulating the equivalence 
oracle Ueq used by QWDNF. The latter step requires estimating the expression E[Mj/xa] to within 
0(e/s) accuracy. This random sample is shared among all boosting stages and all calls to QWDNF. 
The key property exploited here is the oblivious nature of the sampling steps. 

Thus the overall algorithm, if BComb is used as the boosting algorithm, requires 0(s 3 /e + s 2 /e 2 ) 
sample complexity. The best classical algorithm (also based on BComb) has complexity 0(ns 2 /e 2 ). 
Thus, for s = G(l/e), the quantum algorithm is an improvement by a factor of n. 

2 This could be improved to 0(s 2 log(l/e)) rounds if Freund's BComb algorithm is used. 
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Input: Parameters < e, 5 < 1, n, a quantum membership oracle QMQf for a DNF formula /, s 

(the size of DNF /), 

Output: h so that Pr[/ ^ h] < e. 

1. Draw a uniform random sample i? of fL(s 2 /e 2 ) instances x and label using QMQf 

2. 7 <— l/(8s + 4) (weak advantage) 

3. k <— ci7~ 2 e (number of boosting stages) 

4. Mi = 1 (all-one function) 

5. Nq = (all-zero function) 

6. for t = 1, . . . , k do 

7. £ t ^ E^jMtCx)] 

8. if S t < 2e/3 then 

9. break 

10. end if 

11. D t = M t /{2 n E t ) 

12. h t «- QVDSF(n,'ye,S/2k,QMQf • D t ,R) where Pr Dt [h t (x) ^ /(s)] < § - 7. 

13. N t = N t -i + fht-0 

14. M t+ i = [iVt < 0] + (1 - 7)^/2 [JV t > 0] 

15. end do 

16. T = t-1 

17. H(x)^^LiHx) 

18. return h(x) = siga(H(x)). 



Figure 4: The new QHS algorithm. 

6 Lower bounds 

In this section, we prove a lower bound on the query complexity of any quantum PAC learning 
algorithm for DNF formulae. 

Theorem 2 Let s > n/logn. Then any quantum PAC learning algorithm requires Vt{s\ogn/n) 
queries to learn a DNF formula of size s over n variables under the uniform distribution, given 
e < 1/4 and any constant 5 > 0. 

Proof We use a construction given in Bshouty et al. ||. Let t = log s and u = n — t. Consider the 
following class C of DNF formulae over the variable set of V = {x\, . . . , xt} U {yi, . . . , y u }, 

C = I \j x a y a : (y a )a G {o,i} t 
Ue{o,i}' 

where x a = /\* =1 x^\ with the convention x° = Xj and = xj, and for each a £ {0, 1}*, y a is a 
constant (0 or 1) or one of the variables yi or its negation. Each / E C is specified uniquely by 
a word y £ T, s over the alphabet S = {0, 1, yi, yT, • • • ,Uu,Vu}, ke., we may denote / y to be the 
DNF specified by the word y £ S s . By the Gilbert- Varshamov bound, there is a code LcE s with 
minimum distance as of size at least 

[Sjf f (2u + 2) 

E£oG)(|E|-i) fc - V 2 
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We focus on Cl C C where the words y are taken from L. Note that for any distinct y, z G L we 
have Piu[f y / /J = E[/[/y ffiz ] > as/2, where the probability is taken over the uniform distribution 
on V. Letting 2e = as/2, this implies that any two distinct DNF functions f y , f z , where y,z £ L, 
are (2e)-separated. So any (e, <5)-PAC algorithm for Cl must return exactly the unknown target 
function. 

Now let A be any quantum (e, 5)-PAC algorithm with access to a quantum membership oracle 
QMQf associated with a target DNF function /. Suppose that A makes T queries for any function 
/ G Cl- Following the notation in [j?]], let X* be the truth table of the DNF function /, i.e., X* 
is a binary vector of length N = 2 n . Let Ph(Xf) be the probability function of A of returning as 
answer a DNF function h when the oracle is QMQf, for h, f G Cl- By the PAC property of A, we 
have 

• Pf(X f ) >1-S 

It is known that Pj is a multivariate polynomial of degree 2T over X h , for any f,h. Let Nq = 

Y?Zo (?)■ For X e {°i 1 ) JV > let ^ e {°> 1 ) Ar ° be the vector obtained by taking all ^-subsets of [N], 
£ < 2T. The coefficients of P h can be specified by a real vector V h G R N ° and P h (X f ) = V^X f . 
Let M be a matrix of size |Cx| x TVq whose rows are given by the vectors for all h G Cl- Let 
TV be a matrix of size |Cx| x |C^| whose columns are given by the vectors MV g for all g G Cl- 
Observe that the (h, f) entry in the matrix iV is given by Ph{X^). As in 0, we argue that since 
N is diagonally dominant (from the PAC conditions on 5 above), it has full rank. Thus > \Cl\, 
which implies that 

w ->i Cl i>((?^y. 

This implies that 4nT > slog(n)(l — o(l)) which gives T > Q(slogn/n). □ 
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